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Summary of Review 

The new Weighted Student Formula Yearbook 2009 from the Reason Foundation pro- 
vides a simple framework for touting the suceesses of states and urban sehool districts that 
grant greater fiscal autonomy to schools. The report defines the Weighted Student For- 
mula (WSF) reform extremely broadly, presenting a variety of reforms under the WSF um- 
brella. Accordingly, when the report concludes that WSF is successful and should be 
widely replicated, it is difficult to sort through the claims and recommendations. Moreover, 
the approach and recommendations lack critical inquiry, thought, or empirical analysis. 
Perhaps most disturbing is the fact that in a third of the specific districts presented in the 
report, the evidence of success provided predates the implementation of the reforms, and 
the Reason press release makes the outright claim that past improvements are somehow a 
function of yet- to-be-implemented reforms. While the report does provide some reasonable 
recommendations, they are overshadowed by others. Overall, the policy guidance provided 
by the Reason report is reckless and irresponsible. 
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Review 



I. Introduction 

The Weighted Student Formula Yearbook 
2009 from the Reason Foundation, authored 
by Lisa Snell, ^ provides a simple framework 
for touting the successes of urban school 
districts and states that grant greater fiscal 
autonomy to schools. This framework is 
illustrated in Figure 1 . 

The report begins by laying out the key 
principles of Weighted Student Funding 
(WSF), with citations made primarily to the 
2006 Fordham Institute Report, Fund the 
Child (FTC).^ 

Figure 1 

Framework of the Weighted Student 
Formula Yearbook 2009 



Key 

Principles 


Primary source: 
FTC, 2006 






Empowerment 

Benchmarks 


Primary source: 
Ouchi, 2003 






Case Descriptions 
9 WSF* 

5 Pilot 
1 Other 


Primary sources: 
District Documents, 
Media Reports, 
Evaluation Reports 






Local Policy 
Recommendations 


Primary source: 
Case anecdotes & 
original benchmarks 




State Policy 
Recommendations 


Primary source: 
Case anecdotes & 
original benchmarks 


*3 new in 2008-09 



Two main principles dominate the Reason 
rationale: (a) the importance of allocating 
budgets directly to schools within districts. 



based on the characteristics of children in 
those schools, where funding follows the 
child and is based on the needs of the child; 
and (b) the importance of allocating funding, 
as opposed to staff positions, to schools and 
then allowing school level leaders (princi- 
pals) latitude to use that funding as they see 
fit. Similar to a 2007 report published by 
Ohio’s Buckeye Institute, which I also re- 
viewed, the report adds that the principles 
for allocating funding within districts should 
be replicated for all levels, including state 
and federal funding. The report also argues 
for simplicity and transparency. 

The report next lays out a series of “empow- 
erment benchmarks” cited to the work of 
William Ouchi (2003) in Making Schools 
Work."^ These empowerment benchmarks 
provide the outline for the report’s 14 city 
and 1 statewide (Hawaii) reform descrip- 
tions that make up the bulk of the yearbook. 
The report uses these case descriptions as a 
basis for identifying “best practices” for 
school districts implementing or considering 
WSF reforms. 

Weighted Student Formula, which is some- 
times also called Weighted Student Funding, 
is a fiscal resource allocation strategy to be 
used by states when allocating aid to school 
districts or by districts when allocating 
budgets to schools. Several previous reports 
have also attempted to cast WSF as a broad 
net of urban school reform strategies — most 
significantly involving decentralized gov- 
ernance of schools and open-enrollment, 
school-choice programs. 

The new Reason report’s broad definition of 
WSF includes at least the following four 
distinguishable elements: (a) weighted stu- 
dent funding formulas; (b) site-based man- 
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agement; (c) site-based budgeting; and (d) 
school choice, including pilot, magnet or 
charter schools. Notably, this is a much 
broader net than cast in any previous report 
or analysis of which I am aware. 

The report selects a hodge-podge of district 
reform strategies being implemented across 
14 U.S. cities and 1 state. Nine of these re- 
forms are district-wide reforms that include 
implementation of some form of weighted 
student formula — that is, a school-based 
budget allocation formula providing basic 
aid per pupil, with additional weightings, or 
multipliers, based on some committee- or 
administrator-determined set of “need” fac- 
tors. Oakland, for instance, uses a variant of 
this approach, applying a flat foundation 
level per pupil but adding categorical grants 
in place of weights. 

Other district reforms in the analysis have 
little to do with Weighted Student Funding 
at all, nor with whole-district reforms. 
Rather, the Boston, Chicago, Clark County, 
Los Angeles and St. Paul reforms set forth 
in the report involve designating a handful 
of schools within the district to receive 
lump-sum funding and granting them greater 
autonomy in management, contracting and 
hiring. 

These reforms are substantively different 
from and conceptually antithetical to WSF 
reforms. While WSF reforms are designed 
to more equitably distribute fiscal and hu- 
man resources across all schools within a 
district (or even a state), selective pilot 
school programs grant preferential auton- 
omy to some schools with the intent to draw 
resources and creative energy into those 
schools and away from others, generally 
without attention to the plight of others. 
WSFs are intended, in part, to correct for the 
types of inequities that occur when elite 
magnet schools serving advantaged popula- 



tions in urban districts draw resources away 
from disadvantaged students.^ 

The report’s 10 empowerment benchmarks, 
which frame its analysis, also include lim- 
ited emphasis on weighted student formulas, 
per se. The report’s empowerment bench- 
marks may be categorized as follows, with 
adjacent numbers reflecting the order in 
which the report presents the (uncatego- 
rized) benchmarks: 

Table 1: “Empowerment Benchmarks” in 
Weighted Student Formula Yearbook 2009 

Weighted Funding Formula 



1 


School Budgets Based on Students not Staffing 


2 


Districts charge schools actual vs. average 
salaries 


Decentralized Governance 


3 


School choice and open enrollment policies 


4 


Principal autonomy over budgets 


5 


Principal autonomy over hiring 


10 


Collective bargaining relief through flat con- 
tracts 


Recentralized Support of Decentralized Governance 


6 


School level management support 


Reporting & Data 


1 


Published transparent school level budgets 


8 


Published transparent school level outcomes 


9 


Explicit accountability goals 



Notably absent in the report’s Key Princi- 
ples or empowerment benchmarks is the 
original objective of Weighted Student 
Funding-, to increase resource equity across 
schools within districts.^ 

II. Findings and Conclusions 
OF THE Report 

Applying these benchmarks, the report de- 
rives a set of policy recommendations, 
which include the advocacy of cherry- 
picked elements of current policies and prac- 
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tices found in one or more of the 15 in- 
cluded districts. Many of the selected prac- 
tices remain either completely untested or 
are actually refuted in recent empirical stud- 
ies. Most are simply references back to the 
benchmarks provided early in the report. 

For example, the report argues that school 
districts should charge schools for the cost 
of teachers based on their actual salaries 
rather than average salaries. Oakland does; 
therefore it is recommended. These specific 
recommendations are discussed in greater 
detail later in this review. 

III. The Report’s Rationale eor its 
Findings and Conclusions 

In short, the report rationalizes that the 15 
districts studied are all implementing their 
own brand of WSF; all are doing very well 
on one handpicked outcome measure or an- 
other; and all are certainly much better than 
districts not implementing any brand of 
WSF. Therefore, one can necessarily look to 
these districts to identify a list of best prac- 
tices for what all districts and eventually 
states should do. 

IV. The Report’s Use oe 
Research Literature 

For Boston, one of the 15 cases presented, 
the report does reference, with reasonable 
accuracy, the findings of an external evalua- 
tion of the reform. The report notes: 

A 2009 study by the Boston Founda- 
tion that more carefully controlled 
for student characteristics found that 
charter schools are outperforming 
both pilot schools and traditional 
schools. However, students in ele- 
mentary and high school pilot 
schools outperform district schools, 
but middle school pilot students 



score slightly lower than middle 
school students in traditional district 
schools, (p. 31) 

Findings were mixed for the Pilot schools, 
and this is accurately conveyed in the re- 
port.^ 

However, by its own admission, the report 
does not rely heavily on recent empirical 
literature regarding the successes or failures 
of WSF reforms and relies only on “some” 
supporting studies. 

This yearbook utilizes primarily dis- 
trict level documents including dis- 
trict budgets, policy manuals and 
Web site descriptions of school fi- 
nancing systems in addition to some 
supporting studies and newspaper 
accounts, (p. 5) 

Given this approach, readers should under- 
stand two major problems. 

1 . The report neglects large bodies of rele - 
vant literature . 

Because the report sidesteps entirely the 
issue of within-district equity, it correspond- 
ingly avoids the growing body of literature 
that questions whether WSF approaches 
actually achieve greater equity in resource 
distribution across schools within districts. 

For example, one of the studies actually 
cited in the report (for a different issue), by 
Chambers and colleagues (2008), shows that 
between 2001-02 and 2006-07, implicit ad- 
justment for poverty across San Francisco 
schools has backslid, despite implementing 
weighted funding in 2002-03 (p. 74). The 
differences, whether improvements or back- 
sliding, were statistically non-significant.^ 
Further, Baker (2009) found that “widely 
reported WSF success stories provide no 
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more predictable funding with respect to 
student needs than other large urban districts 
in the same state.” 

The report also neglects existing literature 
questioning the efficacy of school-site man- 
agement. (Recall that this report conflates 
WSF with a variety of other reforms, includ- 
ing school-site management.) In a compre- 
hensive review of literature on school-site 
management (SSM) and budgeting, Plank 
and Smith (2008) in the Handbook of Edu- 
cation Finance and Policy present mixed 
findings at best, pointing out that while SSM 
may lead to a greater sense of involvement 
and efficacy, it seems to result in “little di- 
rect impact on teaching behaviors or student 
outcomes.”" 

The report also accepts the rhetoric of Ouchi 
(2003) and others (FTC, 2006) that “power 
to the principals” (control over budgets 
and hiring) has only upsides and cannot pos- 
sibly have any downside. Recent studies of 
principal labor markets and sorting indicated 
that the academic backgrounds of principals 
(i.e., whether they passed certification ex- 
ams, and the nature of their undergraduate 
and graduate training) are highly inequitably 
distributed across schools, both within and 
across districts. Related research shows 
that principals with stronger academic back- 
grounds are more likely to recruit and retain 
teachers with stronger backgrounds when 
granted autonomy to do so. The inverse also 
holds true (weaker principals, weaker teach- 
ers). As such, the current inequitable distri- 
bution of leadership could be harmful for 
high-poverty, high-minority schools under 
highly decentralized systems. 

2. The report neglects disagreeable findings 
in the literature it does cite. 

Most interestingly, the report fails to ac- 
knowledge findings in the studies it does 



cite when those findings disagree with the 
original benchmarks. For instance, one of 
the most intriguing findings of the recent 
American Institutes for Research'^ evalua- 
tion of the Oakland and San Francisco re- 
forms is that Oakland’s use of actual salary 
buy-back has not in fact resulted in im- 
proved distribution of teachers (as regards 
teaching qualifications). 

Despite Oakland’s additional incen- 
tive to retain newer teachers at 
higher-poverty schools, on average, 
San Francisco showed progress to- 
ward closing the experience gap 
whereas Oakland did not.^^ 

V. Review of the Report’s 
Methods 

The yearbook does not present itself as a 
research study. However, it does present a 
framework, and that framework is deceptive. 
The reader is led to assume that these 15 
districts are implementing a similar strategy 
and that are all showing better outcomes 
than they otherwise would have, had they 
not implemented this reform strategy. 

As noted previously, however, no single 
reform strategy is addressed. No uniform 
measure or even approach to measurement 
of outcomes is used. In many cases, no 
comparison groups are included, with or 
without controls for student population dif- 
ferences. Where convenient, the report uses 
average performance on state assessments, 
either compared to the district’s own previ- 
ous performance (Hawaii) or, in a handful of 
cases, compared to state averages or other 
urban districts (Oakland). But convenience 
in other cases apparently led to the use of 
changes in performance instead, or changes 
in achievement gaps, depending on which 
shines the best light on the district being 
discussed. Honest research begins with a 
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justified approach; it does not engage in an 
ad hoc search for ways to present results in 
the best light. 

One noteworthy example is the report’s 
choice to point out that Oakland has shown 
more improvement than all other large Cali- 
fornia cities. This comparison group in- 
cludes San Francisco, which fell in the mid- 
dle of the improvement pack in the graph in 
the Oakland section of the yearbook. But 
when one turns to the section on San Fran- 
cisco, one sees San Francisco described as 
outperforming (level of performance) large 
urban districts for seven years running, and 
no reference to the graph used to show Oak- 
land’s success. 

Most problematic is the fact that in five of 
the 15 cases discussed — one third — outcome 
successes mentioned actually occurred prior 
to the implementation of WSF or SBB/SBM 
(see Appendix A). For example, the report 
commends New York City for winning the 
2007 Broad prize, which it did the year be- 
fore its Fair Student Funding policy was 
implemented. The report might arguably 
attribute this success to mayoral takeover, 
which began in 2002. But this is the WSF 
Yearbook, and even the expansive definition 
of WSF used in the report did not encom- 
pass mayoral takeovers. 

Similarly, the report commends Hartford for 
raising test scores in 2008, the year before 
implementing WSF, and it commends Den- 
ver for making strong improvements be- 
tween 2005 and 2008, whereas WSF was 
implemented in 2008-09. These successes 
lead one to question why these districts 
would want to implement WSF and risk 
undoing their prior achievements. 

The most egregious claim of retroactive 
causation appears in the press release for the 
report: 



The results from districts using stu- 
dent-based funding are promising. 
Prior to 2008, less than half of Hart- 
ford, Connecticut’s education money 
made it to the classroom. Now, over 
70 percent makes it there. As a re- 
sult, the district’s schools posted the 
largest gains, over three times the 
average increase, on the state’s Mas- 
tery Tests in 2007-08 (emphasis 
added). 

Yet, the report itself states that Hartford only 
began implementing WSF in 2008-09, and 
only expected to achieve the 70 percent tar- 
get of available resources allocated to 
schools and classrooms by 2009-2010 (p. 
61). It is difficult to conceive of any defense 
for Reason’s claims. 

Appendix A to this review provides a tabu- 
lar summary of the selected outcome evi- 
dence used in the report. 

VI. Review of the Validity of the 
Findings and Conclusions 

The report’s general conclusion is that WSF, 
however defined for any given district, is 
successful. This success is evidenced by 
invariably positive outcomes, albeit on 
widely varied measures. Therefore, the re- 
port concludes, selected elements of WSF 
should be implemented everywhere. 

However, the initial benchmarks provided, 
as well as the “best practices” recom- 
mended, exist in a vacuum of critical in- 
quiry, thought, or empirical analysis. When 
identifying best practices, the report latches 
on to a variety of potentially problematic 
and untested elements of policies adopted 
across the 14 cities and 1 state, without any 
critical analysis. 

The report also seems enamored with the 
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possibility of providing weights for high 
performance or giftedness, partially bal- 
anced by weights for low performance for 
incoming students, and elimination of 
weights for children from economically dis- 
advantaged backgrounds. The presumption 
(based on Baltimore policies and anecdotal 
evidence) appears to be that policies that 
provide a “U” shaped weighting, high for 
incoming low performers and also high for 
high performers, will encourage schools to 
turn low performers into high performers. 
More likely, however, is the possibility that 
schools that serve clusters of the most ad- 
vantaged children in larger urban districts 
will receive disproportionate resources, at 
the expense of schools with more “average” 
populations and higher-need populations. 
These systems may become more and more 
regressive (that is, reflecting a negative rela- 
tionship between economic disadvantage 
and resources) over time. 

While new policies in Hartford and Balti- 
more warrant investigation, a substantial 
track record of inequities and gaming related 
to “gifted” child weights already exists. In a 
previous Think Tank Review, I found: 

One possible explanation for the lack 
of poverty-related support in Cincin- 
nati is that the district includes a 
weight on gifted students (larger than 
the poverty weight), and across ele- 
mentary schools in the district, the 
correlation between gifted identifica- 
tion rates and poverty is -.88.^^ 

Further, in a series of state-level analyses of 
gifted-education funding. Baker and Reva 
Friedman-Nimz find that many states spe- 
cifically politicize gifted-education funding, 
using it to drive more resources to otherwise 
less needy schools and districts. 

The report specifically advises against allo- 



cating weighting based on poverty, arguing 
instead in favor of weighting for low 
achievement, or “low-scoring students. ”(p. 
139) However, allocating resources on the 
basis of low achievement itself may be far 
more problematic than using a reasonable 
proxy like poverty. Baltimore and Hartford 
appear to protect against the possibility of 
increasing funding by increasing failure. The 
report, though, fails to consider that if 
schools raise the achievement of incoming 
poor students or incoming low performers 
(before they bring them up to the point of 
giftedness, of course), those schools will 
lose the funding that allowed them to pro- 
vide the programs, staff and opportunities to 
improve student performance. Yet, the un- 
derlying out-of-school factors that affect not 
only the child’s starting point but also an- 
nual progress (through opportunities outside 
of school, including summer learning) will 
not have changed.^° 

As noted, a likely outcome of the report’s 
recommendations for rewarding schools 
serving high performers and children identi- 
fied as gifted, and for eliminating poverty 
weighting, will be the advent of more re- 
gressive within-district resource allocation 
formulas than have been seen to date. Baker 
and Green have shown that a handful of 
states have already mastered the tricks-of- 
the-trade of inequitably distributing financial 

resources to school districts on a presumed 
21 

basis of need. 

VII. Usefulness of the Report 
FOR Guidance of Policy 
AND Practice 

Unlike an earlier report from the Fordham 
Institute, which showed increased considera- 
tion for the complexities of WSF reforms 
and more thoughtful integration of state and 
district remedies, the WSF Yearbook is a 
major step backwards. The report haphaz- 
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ardly aggregates a multitude of discrete pol- 
icy issues under an umbrella labeled as WSF 
and deceptively suggests that all related 
policies are necessarily good — even going 
so far as to credit those policies for im- 
provements that took place before the poli- 
cies were implemented. The report then irre- 
sponsibly recommends untested, cherry- 
picked policy elements, some of which may 
substantially undermine equity for children 
in the highest-need schools within major 
urban districts. Additional “best practice” 
recommendations range from reasonable to 



innocuous, including the need for greater 
transparency and clearer public reporting of 
school site budgets, improvements to state 
data systems for tracking school site expen- 
ditures, providing support to principals 
through the process of moving toward site- 
based budgeting and management, and pro- 
viding safeguards and required intervention 
strategies for schools with continued lagging 

■y'l 

performance. Sadly, those reasonable rec- 
ommendations are overshadowed by others. 
Overall, the policy guidance provided by the 
Reason report is reckless and irresponsible. 
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Appendix: Reforms Investigated and Evidence of Success 



City or State 


Implemented 


Retroactive 


Instantaneous/ 


Compared 


External 


Actual 






Causation 


Maintenance 


to other 


Cited 


Statistical 






Effect^ 


Effect^ 


than own 


Research 


Controls'* 










past out- 















comes? ^ 







Weighted Student Funding (9 districts) 
2008-09 



Evidence of “Baltimore's Maryland School Assessment Scores increased in 2008.” (p. 16) 

Success 

(select 

quotes and 

summaries) 






Evidence of “Cincinnati continues to be one of the leaders among Ohio’s urban school districts in perform- 

Success ance. The district is tops among these urban city school systems in the number of report card 

indicators earned (nine versus the next highest urban school system, Columbus, with six) and is 
second only to Akron in its Performance Index Score.” (p. 43) 




* In other words, cases where the report credits WSF for successful outcomes that occurred before WSF was 

implemented. 

^ Does the outcome evidence include claims of improved outcome that occurred concurrent with implementa- 
tion - before implementation would have been completed and could have had any measurable effects 
(Instantaneous)? Maintenance effect refers to those cases where the report explains that the district 
continued to improve, in many cases at a rate of improvement similar to improvement at the beginning 
of, or prior to the reform. 

^ Many cases address performance outcomes only with respect to the district’s own past performance but do not 
explain, for example, whether the district’s own performance gains are better or worse than those of 
other districts. 

* This column addresses whether any attempts were made to compare effects of the reforms on otherwise simi- 

lar (randomly selected or with statistical controls) students, in any of the analyses, internal to the report 
or externally cited evidence. 
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City or State 


Implemented 


Retroactive 


Instantaneous/ 


Compared 


External 


Actual 






Causation 


Maintenance 


to other 


Cited 


Statistical 






Effect^ 


Effect^ 


than own 


Research 


Controls'* 










past out- 
comes? ^ 







Evidence of “From 2005 to 2008, Denver students made strong improvements in reading, math, writing and 
Success science.” (p. 56) 




Evidence of “Flartford schools significantly raised scores on both the 2008 Connecticut Mastery Test and the 

Success 2008 Connecticut Academic Performance Test this year — the first increase since 2001, accord- 

ing to preliminary results released to the district by the State Department of Education.” (p. 66) 




Evidence of Report compares Hawaii against prior performance in Hawaii on NAEP (p. 74) 
Success 




Evidence of Varied anecdotal evidence on numbers of schools meeting standards, numbers of schools im- 
Success proving passing rates, advanced placement courses offered and participation rates, (p. 82) 
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City or State 



New York 
City 



Implemented 



Mayoral con- 
trol in 2002. 
Phase in of 
WSF began in 
2007-08. 



Retroactive 

Causation 

Effect^ 



Instantaneous/ 

Maintenance 

Effect^ 



Compared 
to other 
than own 
past out- 
comes? 



External 


Actual 


Cited 


Statistical 


Research 


Controls'' 




Evidence of • NYC won the 2007 Broad prize (p. 96) for most improved. 

Success • In 2008, NYC elementary and middle school students made substantial progress at every 

grade level in English language arts and math since 2007, outpacing statewide gains. 

• Performance significantly up since 2002. Achievement gap narrowing since 2002. (p.97) 

• Impressive gains on 2007 NAEP, compared to 2005. (p. 97) 

• Numerous additional comparisons of gains from 2002, or 2003 to 2007. 



Poudre 
School Dis- 
trict, Et. 
Collins, CO 




2007-08 







Evidence of • “On 2008 Colorado Student Assessment Program students continued to perform higher 
Success than students state-wide in all 27 areas.” 

• “district-wide averages remain well ahead of state averages...” 

• “Proficiency scores improved or remained the same...” (p. 121) 



Mixed/Undefined Auproach (one district) 




Oakland 2004 expanded 

to all schools 


Y 



Evidence of 
Success 




Oakland demonstrates the largest 4 year API gains among large CA Urban Unified Districts 
(from 2004-2007, 2008). 



Pilot Autonomy Programs (5 districts) 



I 



LA Belmont 


Phase in be- I 


Pilot 


ginning 2007- 1 





08 J 




Evidence of Report provides anecdotal discussion of High School for the Arts 
Success 
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City or State 



Boston Pilot 




Implemented 



Retroactive 

Causation 

Effect^ 



Instantaneous/ 

Maintenance 



1995-96 




Compared 
to other 
than own 
past out- 
comes? ^ 






External 

Cited 

Research 



Actual 

Statistical 

Controls'* 



Evidence of Report cites; Informing the Debate: Comparing Boston’s Charter, Pilot and Traditional 
Success Schools, The Boston Foundation, January 2009: 

http://www.tbf.org/UploadedFiles/tbforg/Utilitv Navigation/Multimedia Library/Reports/ 
InformingTheDebate Final.pdf. 




Evidence of Report cites: Charter Schools Performance Report 2007-2008, Chicago Public Schools, 
Success http://www.ren2010.cps.kl2.il.us/docs/ONS%20perf%20report%202-25 FINAL.pdf . 



I 



Clark County 


2006-07 (17 


Empower- 


schools in 


ment Schools 


2009-10) ** 



Evidence of Report notes average test scores of 4 schools higher than in previous year. (p. 48) 
Success 




Evidence of “Overall Saint Paul public school students made gains across the board on state-wide tests in 
Success 2008. Yet, the district still scores lower than state averages and struggles with large achieve- 

ment gaps between subgroups.” (pp. 128-129) 



*Temporarily suspended in 2009 

**NCES CCD2006 includes 325 CCSD Schools 
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